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2 

Advisement and Management Strategies 
as 'Design Variables in Computer-Assisted Instruction 

Control of the amount and sequence of instructional stimuli has been a 
recurring problem in the design of computer-ajssisted learning environments. 
Methods of control have ranged from those in which the learner plays a 
direct role in decision-making to more highly sophisticated adaptive systems 
(Tennyson & Rothen, 1979) . The adaptive systems includef- processes for 
assessing both the student's skills (for example, general aptitudes, prior 
achievement, on-task learning progress) and the characteristics of the learning 
task (difficulty level, content structure, conceptual attributes) so that an 
initial instructional program can be continuously adjusted to meet on-task 
student learning needs. However, instructional research {DiVesta, 1975) 
and applied projects (Steinberg, 1977) dealing with variables of learner 
control (using rather large or complex learning tasks) have failed to 
demonstrate that students can make and carry out decisions of content 
element selection and personal assessment. It would therefore appear that 
program controlled management systems are necessary for effective computer- 
assisted instruction. 

Given poor student performance with learner control systems consisting of 
a complex content structure and demanding greater prerequisite knowledge, 
Tennyson and associates designed (Rothen & Tennyson, 1978) and tested 
(Park St Tennyson, in press; Tennyson & Rothen, 1977; Tennyson, Tennyson, & 
Rothen, in press) the Minnesota Adaptive Instructional System (MAIS), 
which uses a Bayesian statistical method to integrate (a) assignment of a 
specific treatment based on a premeasure of cognitive ability; (b) an initial 
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amount of instructional support based on a pretest measure of prior achieve- 
ment; and (c) adjusted amount of instructional support and sequence based on 
on-task learning need. Our first purpose in this study was to test the MAIS 
against a learner control strategy (one in which the student decides when 
to terminate instruction and begin the posttest) . Based on the findings of 
the Tennyson and Rothen fl977) study, which showed that the MAIS management 
strategy was significantly more effective than a nonadaptive management 
strategy, we hypothesized that adaptive control would be more effective 
than learner control in helping students to learn concept tasks - 

While adaptive control systems do make use of modern computing power 
(Johansen & Tennyson, in press) , ',:hey do not take into consideration the 
educational goal of individual responsibility for learning and consequent 
intellectual development. To address this goal, instructional program 
strategies have attempted to provide students with post hoc advice about 
possible remedial instructional help (Bunderson, Note 1) . We propose, 
however, that if students are given meaningful information (advisement) 
on- task about their own learning development, their own cognitive strategy 
may further refine the diagnosis and prescription made by an adaptive 
management system. Thus, advisement as used in this study is a form of 
information that consists of diagnostic and prescriptive data generated 
from the MAIS management control system and given to students during the 
learning process. Operationally, advisement implies several things. First, 
at the start of their instruction, students are advised of (a) their initial 
level of knowledge compared to the desired learning criterion (diagnosis) , 
and (b) the amount and sequence of instruction necessary for them to obtain 
the objective (prescription) . Second.^ students are continuously advised while 
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on- task of their learning development (updated diagno', Cid the instructional 

needs (updated prescription) necessary for task mastery. 

For the independent variable of advisement, we hypo.^ \ebized that student 
performance with advisement would be improved over a /entional computer- 
assisted instructional program that lacked advisement. We further hypothesized 
that a learner control strategy with advisement would not only be as 
effective in student acquisition of the learning task as the adaptive control 
strategy (that is, suudents in both these strategy conditions would surpass 
the criterion of mastery) but also that it would be more efficient in terms 
of student on-task learning time. An additional hypothesis was that the 
interaction of learner control with advisement would be the best treatment 
(of the four treatment conditions resulting from the two independent variables) 
in terms of both performance and time on-task, whereas the learner control 
without advisement condition would be the least effective (that is, students 
would not reach mastery) . 

Method 

Subjects 

Participants {N = 139) were twelfth grade male and female students from 
a psychology class at Eisenhower Senior High School in Hopkins, Minnesota. 
Students were assigned randomly to one of four treatment conditions as they 
appeared for the experiment. They understood that they would be given 
credit for participation and that their teacher would grade their posttest 
scores. This contingency was included to stimulate an actual classroom- 
related incentive missing from most learner control studies. Without such 
a contingency, as Felix brod and O'Leary (1974) have shown, students in 
learner control stituations tend to terminate early and thus learn less. 
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A strong incentive to master the material — in this instance,, a course 
grade — provides a better coirpsrison between a program control condition, 
which relies on a relatively captive audience, and a learner control condition. 
Learning Program 

Coordinate concepts selected for this study — positive reinforcement, 
negative reinforcement, positive punishment, and negative punishment — were 
drawn from the field of psychology (Tiemann, Kroecker, & Markle, Note 2) 
and developed by Tennyson, Tennyson, and Rothen, (in press) . Three siibordinate 
concepts — stimulus, aversive stimulus, and attractive stimulus — were included, 
in addition to a superordinate concept that dealt with the consequences of 
behavior resulting from the stimulus. Examples used in the learning program 
and accompanying tests were written according to the concept design strategy 
developed by Merrill and Tennyson (1977) . Of 88 examples in the learning 
program, 40 were used in the instructional lessons (10 per concept) , 24 
in the pretest, and 24 in the posttest. 

Using Tennyson's (in press) revision of the Merrill and Tennyson (1977) 
procedures for designing concept learning lessons, the experimental learning 
program was developed. First, a printed booklet presented the instructional 
directions and the concept definitions with one prototype example per concept. 
The directions explained that the purpose of the definitions was to help the 
student understand the critical attributes of each concept, whereas the 
prototypes were to be used during the instructional presentation to compare 
and contrast with the instructional examples. Second, a computerized 
instructional program presented the examples in two presentation forms — 
expository and inquisitory (Tennyson, Chao, & Youngers, in press) . In the 
expository presentation form, one example of each concept was displayed 
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and students were directed to study how each of these examples differed 
from the prototype examples. After studying the four expository examples, 
students received their remaining examples in the inquisitory presentation 
form. Directions for the inquisitory presentation from instructed students 
to use the prototype examples when determining the correct classification of 
each example. (This procedure helps a student learn to use the prototype in 
developing the intellectual skills of generalization and discrimination.) 
After each response, students received confirmation on whether their answer 
was correct or incorrect. Students in the two advisement groups also received 
an update on the number of examples still needed for mastery. 

To validate the learning program, we used a formative evaluation procedure 
for assessing instructional materials (Tennyson, 1978) . Several siibject 
matter experts first reviewed the definitions and instances. Then, after 
appropriate revisions of the definitions and instances, a one-to-one tryout 
of each learning program was conducted with six randomly selected students 
from the sample population. This was followed by simulation tryouts of 
each treatment condition (six students per treatment) . Final refinements 
on the learning program and computer software were made from this tryout. 
Minnesota Adaptive Instructional System 

To study the management strategy variable, we used for the adaptive 
control strategy the computer-based Minnesota Adaptive Instructional System 
developed by Rothen and Tennyson (1978) . This computer management system 
determines the number, of examples each student receives from three parameter 
values: achievement level, a mastery criterion (.7), and loss ratio (1.5) — 
which is defined as the disutilities associated with a false advance compared 
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to a false retain decision. The estimate of the student's ability to learn 

a concept was characterized in probabilistic terms. From the initial 

achievement level, which was determined by the pretest score and the other 

two parameter values, the probability was used to decide the initial number 

of examples per concept that the student needed. This probability figure 

was adjusted according to the student's on-task performance level, then 

the prescribed number of examples was modified. Student performance on each 

concept was calculated separately with a criterion level set at 1.0 on the 

initial assessment. That is, if the student answered all six examples 

of any concept correctly on the pretest or the initial part of the learning 

program, he or she received no more examples of that concept unless it was 

needed for discriminating coordinate concepts. (For a complete review of 

this response-sensitive procedure, see Tennyson and Park, 1980; or Park 

and Tennyson, in press.) If the students failed to achieve total mastery- 

on the initial assessment, the criterion level was adjusted to suggest a 

prior distribution slightly greater than .5 to the region above the criterion 

level: P = (tt > tt /x,n) > .5, where tt is the objective's criterion level, 
— o_ — — 0^ 

TT is the student's true achievement level, n is test length, and x is the 
student's score (Tennyson & Rothen, 1979). 
Treatment Programs 

The two independent variables of management strategy (adaptive control and 
learner control) and advisement (with and without) were tested with a 
pretest-posttest, two-way factorial design that involved four treatment 
groups. In the adaptive control strategy, the number of instances presented 
to each student was based on the student's pretask and on-task performance 
in relationship to the learning objective, and the sequence of instances 
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was presented according to the student's response pattern to the given 

example. Following the pretest, students in the adaptive control condition 

were given program directions and informed that they would receive a posttest 

at the conclusion of the instruction. In the learner control strategy, the 

students themselves decided whether to continue receiving examples (and if 

so, which concept they wanted to see next) or to go to the posttest. 

Students were informed in the program directions that they had complete 

control on the amount and sequence of instruction. 

For the second independent variable, advisement, the two conditions 

(with and without advisement) were operationally designed as follows: in 

the advisement condition for adaptive control, students were informed, 

« 

following the pretest and after each response, of the number of examples 
needed to reach mastery (diagnosis and prescription information determined 
by the MAIS). Program directions for the adaptive control strategy informed 
students that advisement was determined according to their individual 
learning development in relation to mastering the concepts; in the learner 
control strategy, they were told that it would aid them in deciding the amount 
and sequence of instruction. The adaptive control condition without advise- 
ment did not provide students with information on their learning progress, 
nor did the learner control condition without advisement provide students 
with diagnostic help. Four computer-based instructional treatment programs 
were developed from these four conditions: 

Program 1 . Students hac^ control over the amount and sequence of instruction. 

Advisement was given following the pretest and updated after each response 

(Group 1: learner control with advisement). 
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Program 2 ; Control of the amount and sequence of instruction was determined 
by the computer-based management program. Students were advised of their 
learning progress at the completion of the pretest and after each response 
(Group 2: adaptive control with advisement). 

Program 3 ; Students selected the amoiint and sequence of instruction but 
without advisement on their learning needs (Group 3: learner control 
without advisement) . 

Program 4 : The amount and sequence of instruction were controlled by the 
computer-based management program, and no advisement was given students 
(Group 4: adaptive control without advisement) . 
Facilities 

The experiment was conducted in a small room in the r.iath-science resource 
center at Eisenhower Senior High School. Two ADDS 780 cathode ray teletype 
computer terminals were used for the study. Each terminal, operating at 30 
characters per second, was connected on-line by telephone to a Control Data 
6400 computer at the University of Minnesota. 
Procedure 

As students reported for the experiment, each was assigned to a treatment 
program. The experimenter turned on the terminal and entered each student's 
treatment program number. After receiving direction on operating the terminal, 
students were first administered a 24-item pretest. When the pretest was 
finished, they received a print copy of the four concept definitions and 
prototype examples from the experimenter and told that they should refer 
to these definitions ant. prototype examples during the learning program. 
After studying the definitions and prototype examples, students raised their 
hands to indicate readiness to study the examples in the learning program. 
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The experimenter entered the appropriate command on the terminal for students 
to begin the learning program. After a student classified an example in 
the learning program, he or she received feedback on whether the classification 
was correct or incorrect. When each student finished the learning program, 
the experimenter took the definition and prototype example sheet and entered 
the appropriate command on the terminal for the posttest to begin. All student 
entries were single-letter alphanumeric responses to multiple-choice questions. 
The tests and learning program required no other entries by the student. 
After the students had finished, the experimenter thanked them and they left 
the experiment room. Others were then signed on to the terminal. 



The data analysis consisted of a multivariate analysis with univariate 
tests on each dependent variable followed by mean comparison tests (Student- 
Newman'-Keuls) . Dependent variables included the correct score on the posttest, 
learning program time (the measured time period in which students interacted 
with the learning task, excluding pretest or posttest times) , and number of 
learning program examples. The tests for homogeneity of regression of 
within-class and 'between-class linearity were nonsignificant (£> .05). 

For the multivariate test, we used as dependent variables posttest score 
and time on- task. The main effect of management strategy was significant, 
U (1,^, 137) = .24, £< .001. The test on the second main effect, advisement, 
was likewise significant, U (1, h, 137) = .95, £< .05. The interaction test 
between the two independent variables was nonsignificant (£ > .05). Following 
are the univariate test results on each of the dependent variables. 



Results 
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Posttest Correct Score 

The analysis of variance on the posttest correct score (Table 1) showed 
a difference between the two management strategies, F (1, 135) = 24.07, 
£ < .001. Students in the learner control condition (M = 16.9, 70% correct) 
had a posttest score 4 points lower than students in the adaptive control 
condition (M = 21.1; 88% correct). For the main effect of advisement, 
the F test was significant, F (1, 135) = 16.54, £ < .001, with the group 
receiving advisement (M = 20.8; 86% correct) having scores more than 3 points 
higher than groups receiving no advisement (M = 17.3; 72% correct). The 
Student-Newman-Keuls multiple range test was used to compare posttest 
correct mean score differences betw-;cn the four groups. At the .01 level. 
Group 3 (learner control without advisement) had the lowest posttest score, 
while the other three groups were equal. All groups achieved mastery except 
this one. 

Criterion for mastery was set at .7 on the posttest. Group 1 (learner 
control with advisement) , Group 2 (adaptive control with advisement) , and 
Group 4 (adaptive control without advisement) achieved mastery at the .82, 
.90, and .86 levels — 12 to 20 percent above criterion. Group 3 did not 
reach criterion, falling short by 13 percent (.57). 



Insert Tcible 1 cibout here 

Time On-Task 

Average time spent on the pretest was 7.6 min; time spent on the posttest 
averaged 9.3 min. No significant differences appeared be i^een groups in the 
pretest and posttest times (£> .05). The univariate test on the main effect 

^ ? 
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for management strategy was significant, F (1, 135) = 89.18, £ < .001. 
Students in the two learner control groups spent 6 min (M = 10.0 min) less 
than those in the adaptive control groups (M = 16.1 min^ . For the main 
effect of advisement, the £ test was likewise significant, F (1, 135) = 8.76, 
£ < .05, with the two groups receiving advisement spending 2.3 min more on 
task (M = 14.3 min) than those groups not receiving advisement (M = 12.0 min). 
A comparison of mean time differences for the four groups by the Student- 
Newman-Keuls multiple range test showed that Group 3 spent significantly 
less time (£ < .01) than Group 1. By the same token, Group 1 spent less time 
(£ < .01) than Group 2 or Group 4, both of which equalled (£ > .05) . 



Insert Table 2 about here 

Number of Examples 

The difference in learning time is directly related to the number of 
examples presented in the instruction. The analysis of variance test on 
number of examples for the management: strategy main effect was significant, 
F^ (1, 135) = 168.37, £< .001, with the learner control groups using a mean 
difference of twelve fewer instances (M = 15.2) than the adaptive groups 
(M = 27.0). For the main effect of advisement, F (1, 135) = 35.18, £< .01, 
the groups receiving advisement (M = 23.5) used an average mean of five more 
examples than the groups without it (M = 18.7). The Student-Newman-Keuls 
multiple range test showed that Group 3 studied the fewest number of examples 
(£ < .05), Group 1 the next least amount (£<.05), and Groups 2 and 3 (£> .05) 
more than Groups 1 and 3 (£>.01). In other words, while students in the 
learner control without advisement condition used, on the average, only 
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25 percent of the possible examples, the learner control with advisement 
condition used 51 percent. This was 17 percent less than the two ' adaptive 
control conditions (both at 63 percent) . 

Discussion 

Research on learner control variables has not produced instructional design 
variables that are generic- That is, learner control seems to be a useful 
management format once the correct contingency is identified, but this seems 
to be successful only in highly limited and well-defined occupational areas 
(Steinberg, 1977). Too often, contingencies associated with school-related 
learning, such as grades, praise, or rewards, vary in relationship to 
individual variables, such as sex, age, race, and home environment- This 
frequently results in variables and conditions too confusing for practical 
application or theoretical development- One purpose of this study was to 
introduce a variable to the basic computer-assisted learner control manage- 
ment strategy unlike that of previous research variables- It dealt with 

/ 

actual on-task learning development — advising students of both their 
learning progress (diagnosis) and their individual learning need (prescription) 
to help them master the learning objective- Students would thus have meaningful 
information on which to make judgments about the amount and sequence of 
instruction. 

As operationally defined, the variable of advisement was highly significant 
in providing students in the learner control condition with meaningful 
information with which to make appropriate decisions about acquisition of 
the coordinate concepts- On the posttest, students in the learner control 
with advisement condition (Group 1) did as well as students in the two 
adaptive control conditions (Groups 2 and 4, each over 80 percent correct) . 
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The significance of this result is apparent in contrast to performance in 
the learner control without advisement condition (Group 3) , in which students 
responded to only 58 percent of the posttest items correctly (identical to 
the pretest) . This outcome of the learner control without advisement condition 
is consistent with previous research (Tennyson, in press), which has shown 
that even with a strong contingency such as a grade, students learn little 
/ from instruction; research has furthermore, shown that no matter what level 
of on-task attainment is reached, all students leave instruction at approxi- 
mately the same time (see Tennyson & Rothen, 1979) . 

The dependent variable of time is important to consider in the study of 
learner control management strategies because students in a learner control 
condition consistently leave instruction before mastering the objective 
(Tennyson, Tennyson, & Rothen, in press). In contrast no this basic finding, 
students in the" learner control group which received advisement stayed on-task 
long enough to obtain mastery. In fact, they were on-task approximately 
39 percent longer than the students in the conventional learner control 
condition. It was our thesis that if students in a learner control strategy 
were given advisement in the form of adaptive diagnostics and prescriptions, 
they would master the objective in less time and use less instruction than 
in a program-controlled adaptive system. The assumption was that the cognitive 
strategy that students used in learning would further refine the adaptive 
information. The findings support this notion. The two conditions using 
the adaptive information (adaptive control and learner-adaptive control) 
had identical posttest-score means, but the learner control with advisement 
condition showed significant decreases in on-task time (22 percent less) and 
amotint of instruction {25 percent less) . 
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In conclusion, a learner control condition can be a valuable instructional 
ananagement system, especially for computer-based instruction, if students 
receive sufficient information about their learning development ~ information 
that continuously shows them what progress they have made toward mastery 
of the objective and provides meaningful advice on appropriate stimuli 
necessary to obtain it. 
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Table I 

Mean and Standard Deviations for Posttest Correct Scores 



Management Strategy- 
Advisement 



Learner Control Adaptive Control 

With Group 1: Group 2: 

M 19.9 21.6 

SD 3.3 3.2 

Without Group 3: Group 4: 

M 13.9 20.6 

SD 3.3 3.7 

Note: Maximum posttest score = 24 
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Table II 
Mean and Standard Deviations 
for Time On-Task and Number of Examples 



Management Strategy 
Advisement 

Learner Control Adaptive Control 

Time Number Time Number 



With 


Group 


1: 


Group 2 : 




M 


12.5 


20.2 


16.0 


26.7 


SD 


5.5 


9.0 


4.4 


5.6 


Without 


Group 


3: 


Group 4: 




M 


7.7 


10.1 


16.2 


27.3 


SD 


6.7 


8.7 


3.0 


4.2 




